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^ (54) Title: METHODS AND SYSTEMS FOR CORRECTING IMAGE MISALIGNMENT 



(57) Abstract: The invention provides methods of determining a correction for a misalignment between at least two images in a 



sequence of images due at least in part to sample movement. The methods are applied, for example, in the processing and analysis of 
^ a sequence of images of biological tissue in a diagnostic procedure. The invention also provides methods of validating the correction 
^ for a misalignment between at least two images in a sequence of images of a sample. The methods may be applied in deciding 
^ whether a correction for misalignment accurately accounts for sample motion. 
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1 . A method of compensating for image misalignment, the method comprising the steps 
of: 

obtaining a sequence of images of aiissue-sample; and 

correcting for misalignment between at least two of the images, said misalignment being 
due at least in part to movement of the tissue sample. 

2. The methocTof claim 1, wherein the correcting step is performed in real time. 

3. The method of claim 1, wherein the correcting step comprises adjusting an optical signal 
detection device used to obtain the sequence of images. 

4. The method of claim 3, wherein the correcting step comprises adjusting a position of a 
component of the optical signal detection device. 

5. The method of claim 4, wherein the component comprises a mirror. 

6. The method of claim 1, wherein the tissue sample is an in-situ tissue sample and wherein 
the misalignment is due at least in part to patient motion. 

7. The method of claim 1 , further comprising the step of applying a contrast agent to the 
tissue sample. 

8. The method of claim 1, wherein the correcting step comprises electronically adjusting at 
least one of the images. 

9. The method of claim 1 , wherein the at least two images are consecutive images. 

1 0. The method of claim 1 , wherein the correcting step comprises the step of filtering a 
subset of data from a first image of the sequence of images. 

1 1 . The method of claim 1 0, wherein the correcting step comprises the step of preprocessing 
the subset of data prior to the filtering. 

12. The method of claim 10, wherein the filtering step comprises at least one of frequency 
domain filtering and discrete convolution in the space domain. 
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1 13. The method of claim 1 0, wherein the filtering step comprises Laplacian of Gaussian 

2 filtering. 

1 14. The method of claim 1 0, wherein the filtering step comprises using a feathering 

2 technique. 

1 15. The method of claim 1 0, wherein the filtering step comprises using a Hamming window. 

1 16. The method of claim 1 , wherein the correcting step comprises computing a cross 

2 correlation using data from two of the images. 

1 1 7. The method of claim 1 6, wherein the computing of the cross correlation comprises 

2 computing a product represented by 

3 F<u,v)F*j(u,v), 

4 where Fi(u,v) is a Fourier transform of data derived from a subset of data from a first image, i, of 

5 the sequence of images, F*j(u,v) is a complex conjugate of a Fourier transform of data derived 

6 from a subset of data from a second image, j, of the sequence of images, and u and v are 

7 frequency domain variables. 

1 18. The method of claim 1 7, wherein the computing of the cross correlation comprises 

2 computing an inverse Fourier transform of the product. 

l 1 9. The method of claim 1 , wherein the tissue sample comprises cervical tissue. 

1 20. The method of claim 1 , wherein the tissue sample comprises at least one member of the 

2 group consisting of colorectal tissue, gastroesophageal tissue, urinary bladder tissue, lung tissue, 

3 and skin tissue. 

1 21. The method of claim 1 , wherein the tissue sample comprises epithelial cells. 

1 22. The method of claim 1 , wherein the obtaining step comprises obtaining the sequence of 

2 images of the tissue sample during application of a chemical agent to the tissue sample. 
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1 23 . The method of claim 1 , wherein the obtaining steg^comprises-obtaining the sequence of 

2 images of the tissue sample after application of a chemical agent to the tissue sample. 

1 24. The method of claim 23, wherein the chemical agent is selected from the group 

2 consisting of acetic acid, formic acid, propionic acid, and butyric acid. 

1 25. The method of claim 23, wherein the chemical agent is selected from the group 

2 consisting of Lugol's iodine, Shiller's iodine, methylene blue, toluidine blue, indigo carmine, 

3 indocyanine green, and fluorescein. 

1 26. The method of claim 1 , wherein the obtaining step comprises obtaining the sequence of - 

2 images of the tissue sample during an acetowhitening test. 

1 27. The method of claim 1, wherein the movement of the tissue sample is relative to an 

2 optical signal detection device and comprises at least one member of the group consisting of 

3 translational motion, rotational motion, warping, and local deformation. 

1 28. The method of claim 1, wherein one or more inmges_ofthe sequence of images comprise ~t 

2 measurements of an optical signal from the tissue sample. 



1 29. The method of claim 28, wherein the optical signal comprises visible light. 

1 30. The method of claim 28, wherein the optical signal comprises fluorescent light. 

1 31. The method of claim 28, wherein the optical signal is emitted by the tissue sample. 

1 32. The method of claim 28, wherein the optical signal is reflected by the tissue sample. 

1 33. The method of claim 28, wherein the optical signal is transmitted through the tissue 

2 sample. 

1 34. A method of validating a correction for an image misalignment, the method comprising 

2 the steps of: 

3 adjusting at least one of two or more images using a correction for an image 

4 misalignment between the two or more images; 
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5 defining one or more validation cells, each of which includes a common area of the two 

6 or more adjusted images; 

7 computing for each of the one or more validation cells a measure of displacement 

8 between the two or more adjusted images using data from the two or more adjusted images 

9 corresponding to each of the one or more validation cells; and 

10 validating the correction for the image misalignment by comparing at least one of the 

1 1 measures of displacement with a threshold value. 

1 35 . A method of validating a correction for an image misalignment, the method comprising 

2 the steps of: 

3 defining one or more validation cells within a bounded image plane; 

4 computing for each of the one or more validation cells a measure of displacement 

5 between two or more images bound by the image plane using data from the two or more images 

6 corresponding to each of the one or more validation cells; 

7 validating a correction for an image misalignment between the two or more images by 

8 comparing at least one of the measures of displacement with the correction. 

1 36. The method of claim 34, wherein the images are images of an in-situ tissue sample, and 

2 wherein the image misalignment is due at least in part to patient motion. 

1 37. The method of claim 34, wherein the images are images of an in-situ tissue sample that 

2 has been treated with a contrast agent 

1 38. The method of claim 34, wherein the one or more validation cells comprise a subset of a 

2 bounded image plane common to the two or more images. 

l 39. The method of claim 34, wherein the two or more images are consecutive images. 
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2 40. The method of claim 38, wherein the one or more validation cells comprise a central 

3 portion of the bounded image plane. 

1 41. The method of claim 3 8, wherein the bounded image plane has an area about four times 

2 larger than the total area of the one or more validation cells. 

1 42. The method of claim 34, wherein the validating step comprises eliminating from 

2 consideration one or more of the measures of displacement for one or more of the one or more 

3 validation cells. 

1 43. The method of claim 42, wherein the eliminating of the one or more measures of 

2 displacement comprises calculating a sum squared gradient for at least one of the one or more 

3 validation cells. 

1 44. A method of compensating for an image misalignment, the method comprising the steps 

2 of: 

3 obtaining a set of sequential images of a tissue sample; and 

4 correcting for a misalignment between each of a plurality of pairs of the sequential 

5 images, the misalignment due at least in part to movement of the tissue sample. 

1 45. The method of claim 44, wherein the tissue sample is an in-situ tissue sample and 

2 wherein the misalignment is due at least in part to patient motion. 

1 46. The method of claim 44, further comprising the step of applying to the sample a contrast 

2 agent. 

1 47. The method of claim 44, wherein the obtaining step and the correcting step are performed 

2 alternately. 

1 48. The method of claim 44, wherein the obtaining step and the correcting step are performed 

2 substantially concurrently. 
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1 49. The method of claim 44, wherein the correcting step comprises determining a correction 

2 for a misalignment between a pair of the sequential images less than about 2 seconds after the 

3 obtaining of the latter of the pair of the sequential images. 

1 50. The method of claim 44, wherein the correcting step comprises determining a correction 

2 for a misalignment between a pair of the sequential images less than about one second after the 

3 obtaining of the latter of the pair of the sequential images. 

1 51. A method of validating a correction for an image misalignment, the method comprising 

2 the steps of: 

3 obtaining a plurality of sequential images of a sample using an optical signal detection 

4 device; 

5 determining a correction for a misalignment between at least two of the sequential 

6 images, the misalignment due at least in part to a movement of the sample; and 

7 validating the correction between at least a first image and a second image of the plurality 

8 of sequential images. 

1 52. The method of claim 5 1 , wherein the sample is an in-situ tissue sample and wherein the 

2 misalignment is due at least in part to patient motion. 

1 53 . The method of claim 5 1 , further comprising the step of applying a contrast agent to the 

2 sample. 

1 54. The method of claim 5 1 , wherein the determination of a correction for a misalignment 

2 between a first and a second image and the validation of said correction are performed in less 

3 than about one second. 

1 55 . The method of claim 5 1 , further comprising the step of: 

2 adjusting the optical signal detection device using the correction. 

1 56. A method of dynamically compensating for image misalignment, the method comprising 

2 the steps of: 
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3 obtaining a sequence of images of a tissue sample; and 

4 correcting in real time for misalignment between at least two of the images, the 

5 misalignment due at least in part to movement of the tissue sample. 
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